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ABSTRACT 



Methodologies vere proposed for combining background 
and aptitude and/or achievement information in an approach to the 
classification problem using the concept of under- and 
overachievement. The methodologies vere applied to independent 
samples from four high school curricula. The results suggest that 
when past achievement and/or aptitudes vere held constant (1) groups 
characterized by overachievement in all four curricula had similar 
backgrounds as defined by their level and pattern of responses to 
five biographical variables, (2) groups characterized by 
underachievement in the four curricula appeared to have similar 
response patterns but had moie variation with respect to level and 
(3) underachieving groups regardless of curricular chioce reported 
that they spent more time on extracurricular technological projects 
than did overachieving groups. To a somewhat lesser extent 
underachievers tended to have fathers who discouraged their going on 
to college and they (the underachievers) also tended to spend more 
time on extracurricular reading. (Author) 
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Cultural Characteristics as a Moderating Influence on Expected 
Achievement within a Curriculum Choice 
(Growth Study Paper // 58) 

Donald A. Rock and Franklin R. Evans 
Abstract 

Methodologies were proposed for combining background and aptitude and/ 
or achievement information in an approach to the classification problem 
using the concept of under- and overachievement. The methodologies were 
applied to Independent samples from four high school curricula. The results 
suggest that when past achievement; and/or aptitudes were held constant (1) 
groups characterized by overachievement in all four curricula had similar 
backgrounds as defined by their level and pattern of responses to five 
biographical variables, (2) groups characterized by underachievement in the 
four curricula appeared to have similar response patterns but had more 
variation with respect to level and (3) underachieving groups regardless of 
curricular choice reported that they spent more time on extracurricular 
technological projects than did overachieving groups. To a somewhat lesser 
extent underachievers tended to have fathers who discouraged their going 
on to college and they (the underachievers) also tended to spend more time 
on extracurricular reading. 



Cultural Characteristics as a Moderating Influence on Expected 
Achievement within a Curriculum Choice^ 

(Growth Study Paper // 58) 

Donald A. Rock and Franklin R* Evans 

With the increased emphasis on efficiency in education today , it 
becomes of great practical concern to know what determines how much a 
student profits from his curriculum choice. It is of considerable im- 
portance to be able to define and describe those measurable characteristics 
of an individual's background that may interact with his achievement in 
his selected curriculum. Such information is, of course, necessary for 
effective and knowledgeable guidance and placement decisions. 

The placement problem is in general concerned with exposing indi- 
viduals to those treatments” from which they will most profit subject to 
certain constraints within the system such as availability of treatments, 
etc. This is the traditional classification problem. In its purest sense 
the problem is unsolvable since in general an individual will appear in 
only one treatment group so criterion information is not available for 
him on all treatments. Thus, we must develop a methodology which enables 
us to make educated guesses about whether or not a person would have done 
better if he were indeed in a different treatment classification. This 
study will propose and empirically test techniques for arriving at com- 
promise solutions to typical classification problems. 

The major objectives of the study reported here were to arrive at 
tentative answers to the following questions: 

(1) If the input with respect to past achievement as measured by a 
battery of standardized tests is held constant (within a curriculum choice) 
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will it be possible to identify subgroups (i.e., groups of individuals 
having homogeneous profiles on background information) which show sig- 
nificantly greater or lesser achievement on the average? 

(2) Are the same background patterns always associated with over- 
achievement and conversely underachievement independent of curriculum 
choice? That is, are the cultural patterns with respect to both level and 
shape associated with individual under- or overachievement invariant 
across curricula? 

(3) If (2) above is not true or only partially true and the attributes 
for success and/or failure are not consistent across curricula, then it is 
hoped that the background profiles of those individuals who do considerably 
less well than one would expect from cheir aptitudes may, however, have 
similar profiles to a success group in another curriculum. For example, 
success may be ”achieved” via many avenues while one’s channels to failure 
may be more or less limited or vice versa. It should be made clear here 
that our terms success and failure within a curriculum are only relative 
and simply reflect whether or not a group’s performance with respect to 
school grades is better (or worse) on the average than would be predicted 
from their inputs on past achievement test scores. 

Method 

It is here that we choose to deviate from the more ’’traditional” 
extreme group research methodology in favor of using taxonomic techniques 
which form ’’natural” groups of individuals which are characterized by 
frequently occurring background patterns as defined by their responses to 
biographical items. That is, it was felt that if one can divide the total 
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sample into homogeneous subclasses of individuals rather than define 
groups which are extreme with respect to performance on some criterion 
of interest, the prognosis for the generalization of any subsequent find- 
ings would be significantly improved. 

Ihus the focus of this study is on the formation of clusters of in- 
dividuals within each curriculum according to the similarity of their 
multidimensional response patterns on biographical data. Then the various 
groups of individuals are examined with respect to their relative level 
of under- and overachievement. 

If the cultural or background patterns associated with under- and 
overachieving groups are found to differ across curricula, the question 
can be asked, "Is there an underachieving group in one or more curricula, 
which is similar with respect to background patterns to an overachieving 
group in another curriculum ? 11 If there is indeed sufficient group over- 
lap, we may expect to find a substantial number of individuals in the 
underachieving group who are more like the overachievers in another 
curriculum than they are like the average underachievers in their own 
group. That is, if we plotted the points for each individual in the 
multidimensional space defined by the background variables, the points of 
many of the underachievers would lie closer to the centroid (locus of 
maximum density) of the overachievers than they would to the centroid of 
their own groups. 

The formation of homogeneous subgroups within curricula was accom- 
plished through the use of a multiple moderator technique (Rock, Barone, 

& Linn, 1967). This technique iteratively searches among a set of back- 
ground variables (up to a maximum of five) for those particular variable^ 
singly or in combination, which define groups which in turn are character- 
ized by differing relationships between criterion and a system of predictor 
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variables. The grouping procedure used in the moderated regression is 
a modification of Ward's (1963) hierarchical clustering procedure applied 
to five background response patterns. The procedure begins with g groups 
and then forms g-1 groups selectively collapsing groups which maximize the 
between to within group sum of square and continues until the researcher 
notices a large increment in the within group sum of squares (indicating 
the combining of two rather dissimilar groups) and/or until the number of 
individuals within the smallest group becomes large enough to provide 
satisfactory future statistical estimates. 

Depending upon the particular objective of the study, any one of 
three objective functions may be used with this technique. Consistent 
with the purpose of this study the objective function was selected which 
was designed to identify particular groups of students characterized by 
within group homogeneity with respect to backgrounds, yet for whom the 
overall regression equation based on a battery of achievement tests re- 
sults in overpredictions or underpredictions. It does this by grouping 
individuals according to similarity of patterns of responses to background 
questions and computing the relationships between the mean grade point 
average (GPA) that had been predicted and the mean GPA that had actually 
been obtained by that group. Subtraction of the predicted values from the 
obtained values yields a mean residual which is an index of the amount of 
overprediction (a negative residual) or underprediction (a positive 
residual) which characterizes that particular group. 

This function enables one to identify the groups which yield the 
largest absolute difference in mean residual values. This will result 
in a system of subgroups, two of which are of particular interest j one that 



is characterized by overachi evemcnt , the other by unde radii cvcmcn t (FInughcr 
& Rock, 1969). 

Having formed clusters of individuals who are characterized by over- 
and underachievement within each curriculum we then need methodology to 
measure the relative similarity or dissimilarity of the background profiles 
of the various groups. Since groups may be formed on the basis of up to 
five background variables, the problem becomes one of estimating the over- 
lap between the multivariate distribution associated with each group. The 
centour concept as described by Cooley and Lohnes (1962) and Rulon, Tiedcman, 
Tatsuoka, & Langmuir (1967) appears to be well suited for this problem. 

The centour score provides a good index of the extent to which an 
individual or group rerembles another group where the second group’s means 
and dispersions are known. The centour method may* of course* be generalized 

to any number of variables. The centour score is the probability of ob- 

2 

taining a larger chi-square ( X )> thus when an individual’s scores are 

compared with the mean scores associated with a particular group with a 

2 

known dispersion matrix, the larger the x > the less likely an individual 
having said scores would be a member of that population. More rigorous 
treatments of this application of the maximum likelihood classification 
procedures may also be found in Rao (1952) and Tatsuoka (1957). 

Samples 

Data for this study were taken from the Study of Academic Growth and 
Prediction (Hilton & Myers, 1967). Four high school curricula were selected 
for analysis. The four curricula were academic, vocational, business, and 
general. In order to get a fairly large sample with complete data cases 
the subjects were selected from three different high schools. The sample 
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sizos within each curriculum were: academic. 550 ; vocational, 354 ; business, 

314; and general, 276. 

Predictors 

The predictors were scores on five of the Sequential Tests of Educa- 
tion Progress (STEP) which were administered when the students were in the 
ninth grade. The five predictor scores included mathematics, science, 
social studies, reading, and writing. The criterion was senior rank in 

class which jas subsequently adjusted (Angoff, 1961) using the individual's 

2 

School and College Aptitude Test (SCAT) scores, in order to reduce 
differences among schools in grading practices which may be due to dif- 
ferences in ability inputs. 

Background Variables 

The background or moderating variables were selected on the basis of 
past research (Rock, 1968) which seemed to indicate their potential for 
acting as measures of motivation. Three of the five variables in particular 
have demonstrated moderator characteristics. One of these variables was an 
"Environmental Participation" scale (Moderator 1) which was constructed 
in an effort to yield estimates of the exposure of the individual to 
typical middle class experiences and environmental stimulation. It is 
hypothesized that poverty of experiential stimulation rather than socio- 
economic class would have the greater moderating effect on the regressions . 
The remaining two "motivation" type variables were standard "family press" 
type biographical items which were concerned with the parent's attitude 
towards school achievement (Moderator 2) and the father's attitude toward 
continuing on to college (Moderator 3) . The fourth biographical variable 
was concerned with the amount of nonschool related reading done. More 




3 



-7- 



spccifically tills scale consisted of items asking about the number of 
books read in the areas of current events, biographies, music, art, poetry, 
science, etc. The fifth (potential moderator) asked about time spent on, 
as well as interest in, home economics, vocational or technological type 
activities such as repairing cars, machinery, etc. 

The moderated regression technique wa3 run independently within each 
curriculum sample. Since the sample sizes were not sufficiently large to 
c/llow splitting into random halves within curriculum, stability of the 
results must depend on whether or not the various success and/or failure 
patterns could be replicated across curricula. 

Results and Discussion 

Table 1 presents an 8 x 8 matrix of centour scores indicating which 
of the over- and underachieving groups are most similar or dissimilar to 
each other based ou their group means on the five biographical variables. 
Within each curriculum of Table 1 there is a column of centour scores for 



Insert Table 1 about here 



the overachieving group and the underachieving group. An entry c,, in 

J K 

the matrix of centours indicates the centour score associated with group k 
in the group j dispersion. For example, c^ indicates that an individual 
having group 1 mean scores, an overachiever in the academic curricula, is 
closer to the centroid of group 3 than 98% of the individuals in group 3. 
Conversely c^ indicates that an individual having group 3 means lies 
closer to the center (centroid) of group 1 than do 89% of the members of 
group 1. More singly, if you wish to know which groups the underachiever 
in the vocational curriculum is most like, you would simply go down column 4 
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noting the largest entries. Thus high values on the off-diagonals in- 
dicate a high degree of overlap between the corresponding groups. Lack 
of symmetry in the appropriate off-diagonal elements, e.g,, c^ 9* is 

due to differences in dispersions for the paired groups. 

A closer inspection of Table 1 shows that in general the overachieving 
groups show considerable similarity with respect to response patterns on 
the background items. That is the overachievers within any one curriculum 
were far more like the overachievers in another curriculum than the under- 
achievers in their own or any other curriculum. In 12 possible comparisons, 
i.e., the over achieving group from each one of four curricula with the over- 
achieving groups in the remaining three curricula, 10 yield centours in excess 
of 80. It was also encouraging to note that with the possible exception 
of the general curriculum, the centour scores indicate that the background 
patterns for the overachievers in any one curriculum were considerably 
dissimilar to those of the underachievers when past achievement is held 
constant. 

In an attempt to present a more familiar index of the extent of 
separation of the under- and overachievement groups within curriculum than 
provided by the centour score, multiple correlations were computed be- 
tween a criterion of group membership and scores on the five background 
variables. The multiple correlations were .82, .88, . 81, and .77 for the 
academic, vocational, business, and general curriculum samples respectively. 

As the centours also indicated the background variables do significantly 
discriminate between the under- and overachiever within curriculum. 

At first glance, there appears to he. m_.con^istent background pattern 
associated with the underachievers. However, as we shall see later on the 
low centours between underachieving groups are primarily due to differences 
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in level rather than in the patterns of responses to the background ques- 
tions. The consistency in the similarity of backgrounds among the over- 
achievers with respect to both pattern and level as compared to the relative 
lack of similarity among the underachievers suggests that the n paths to 
success ” may be somewhat limited while there are more ’’roads ” to under- 
achievement . 

Table 2 shows the centroids or vector of means for each of the over- 
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and underachievers within curriculum. Three of the five background means 
demonstrate a consistent pattern which is replicated over all four curricula. 
That is, for all four curricula when past achievement is held constant the 
underachieving (1) tend to spend more time on nonschool related reading, 

(2) tend to have fathers who discourage their continuing on to college (high 
scores indicate less encouragement), and (3) spend more time on vocational 
or technological type activities. It should be pointed out here that it 
is the multidimensional pattern of the three background variables rather 
than any one alone which differentiates the under- from the overachievers. 

In fact, when individuals are grouped according to the level of extra- 
curricular reading alone, the more active readers tend to overachieve in 
both the academic and vocational curriculum. There is no such clear differ- 
entiation in the general or business curriculum. It is possible that the 
incongruous combination of interest in nonschool related reading while 
operating within a relatively negative family press situation may prove 
somewhat debilitating with respect to achieving the expected academic per- 
formance. It also should be noted here that reading achievement as measured 
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by the STEP reading test has been held constant within curriculum in the 
model used in this analysis. This, of course, suggests that given the same 
reading achievement level, those individuals who report spending consider- 
able time on nonschool related reading in conjunction with the previously 
specif ied pattern on background variables achieve less well than expected. 

As a result of looking at such complex interactions one begins to under- 
stand why our relatively oversimplified standardized test battery prediction 
models may often fall short of the desired accuracy for any one given subset 
of the population of interest. 

Indication of the relative contribution of the three background 
variables to the discrimination between groups characterized by over- and 
underachievement within curriculum were obtained from inspection of the 
standardized partial regression weights from the multiple correlational 
analysis referred to earlier. The fifth moderator, time spent on voca- 
tional or technological extracurricular activities, had a statistically 
significant regression weight in all four curriculum samples. Reported 
time spent on ’nonschool related reading and father’s attitude towards con- 
tinuing higher education were statistically significant in three of the 
four and two of the four curriculum samples respectively. The under- 
achievers appear to spend considerable time on technological activities 
and reading material that evidently are not rewarded in the normal school 
grading practices. The fact that interest in vocational or technological 
activities appears to be the best discriminator between under- and 
overachievers suggests that simple sex differences may be an equally potent 
predictor of under- or overachievement. Thus sex was coded and entered 
into the prediction equation with the five grouping variables. In none of 
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the four curricula was sex found to be a significant predictor. It would 
seem that in this study the extent of one’s interest in technological 
activities is a far more important variable than knowledge of sex group 
membership for predicting under- and overachievement. 

The fact that underachievers tend to report a lack of ’’family press” 
towards continuing education has also been reported in a previous study 
of over- and underachievement (Rock, 1968). A hi,*h level of extracurricular 
activities of a more general nature were found to be related to under- 
achievement in another earlier study by Flaugher and Rock (1969) . Further 
inspection of Table 2 indicates that although underachievers demonstrate 
a consistent pattern with respect to three of the background variables, 
the relative levels vary from curriculum to curriculum. This, of course, 
contributes to the lower centour scores between the underachievers . 

Since most of the overachieving groups looked only like other over- 
achieving groups the attempt to find an underachieving group which looked 
like an overachieving group in another curriculum was not especially 
successful. However, the underachievers in the academic curriculum did 
show extensive overlap with respect to background response patterns with 
the overachievers in the vocational curriculum. That is, inspection of 
Table 1 indicates that the centroid of the academic underachievers is 
closer to the centroid of the vocational overachievers than approximately 
83% of the members of that group. However, any meaningful comparison across 
curriculum would have to include aptitude information in addition to back- 
ground patterns. That is an underachiever in the academic curriculum could 
have the same background pattern as an overachiever in the vocational 
curriculum yet his aptitudes may still be inappropriate. Since there was 
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no opportunity for replication of this finding in this study, these re- 
sults can only serve as a means for future hypothesis generation. 

C onclusions 

Methodologies were proposed for combining background and aptitude and/or 
achievement information in an approach to the classification problem using 
the concept of under- and overachievement. The methodologies were applied 
to independent samples from four high school curricula. The results suggest 
that when past achievement and/or aptitudes were held constant (1) groups 
characterized by overachievement in all four curricula had similar back- 
grounds as defined by their level and pattern of responses to five biographi- 
cal variables, (2) groups characterized by underachievement in the four curricula 
appeared to have similar response patterns but had more variation with re- 
spect to level and (3) underachieving groups regardless of curricular choice 
reported that they spent more time on extracurricular technological projects 
than did overachieving groups. To a somewhat lesser extent underachievers 
tended to have fathers who discouraged their going on to college and they 
(the underachievers) also tended to spend more time on extracurricular 
reading . 

The results also suggest that the "natural" grouping technique pro- 
posed here may increase the possibility of replication from sample to 
sample while not leading to particularly exciting or extreme results with- 
in any one given sample. 
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Footnotes 



^This research was performed pursuant to a grant with the Office of 
Education, U. S. Department of Health, Education, and Welfare. 

^Further information on the STEP and SCAT test batteries can be obtained 
by writing to : 

Cooperative Test Division 
Educational Testing Service 
Princeton, N. J. 08540 
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